Decoding apparatus, decoding method, and program of same

ABSTRACT

An encoding apparatus for encoding progressive image data and interlace image data corresponding to this progressive image data, having a first encoding unit configured to encode picture data composing the interlace image data to generate first encoded data and decoding and recomposing the first encoded data to generate recomposed image data, an up sample processing unit configured to up sample the generated recomposed image data to generate image data having the same resolution as that of the progressive image data, and a second encoding unit configured to encode the picture data composing the progressive image data to generate second encoded data using the generated image data as predictive image data.

TECHNICAL FIELD

The present invention relates to an encoding apparatus for encodingimage data, an encoding method, and a program of the same and a decodingapparatus for decoding the image data, a decoding method, and a programof the same.

BACKGROUND ART

In recent years, for the purpose of digitally handling image data andtransmitting and storing information with a high efficiency at thattime, after the MPEG (Moving Picture Experts Group) system utilizing theredundancy peculiar to image data and compressing it by a discretecosine transform (DCT) or other orthogonal transform and a motioncompensation, encoding apparatuses and decoding apparatuses based on theH.264/AVC (Advanced Video Coding) and other encoding systems (method)having a higher compression ratios have been spreading in bothdistribution of information by broadcast stations etc. and reception ofinformation in general homes.

At present, efforts are being made for standardization by adding afunction of scalability to this H.264/AVC, that is, SVC (Scalable VideoCoding). The present specifications of SVC are put together in the JSM(Joint Scalable Video Model).

In an SVC encoding device, an input image is separated into for exampletwo layers of an upper layer and a lower layer by an image layeringcircuit. Thereafter, the upper layer is encoded by an upper layerencoding circuit, and the lower layer is encoded by a lower layerencoding circuit. Then, the encoded upper layer and lower layer aremultiplexed and transmitted.

The lower layer is also called a “base layer” and is a layer of a lowquality. When decoding only the bit stream of the lower layer, an imagehaving a relatively low quality is decoded. The lower layer includesmore important information as the quality.

The upper layer is also called an “enhancement layer” and is a layer forenhancing the quality and decoding a high quality image. When decodingthe bit stream of the upper layer in addition to the bit stream of thelower layer, it is possible to decode a higher quality image.

In the encoding apparatus explained above, when the upper layer encodingcircuit performs intra-coding, it is possible to use the decoded imageobtained by encoding, then decoding at the lower layer encoding circuitas a predictive image.

DISCLOSURE OF THE INVENTION Problem to be Solved by the Invention

The conventional encoding apparatus performing the layered encodingexplained above is predicated on encoding progressive image data at bothof the upper layer and the lower layer.

However, there are demands for encoding the progressive image data inthe upper layer and encoding interlace image data in the lower layer.

From the above, it is desired to provide an encoding device, an encodingmethod, and a program enabling encoding of progressive image data in theupper layer and encoding of interlace image data in the lower layer whenperforming layered encoding.

Further, it is desired to provide a decoding device, a decoding method,and a program enabling decoding of progressive image data encoded in theupper layer and interlace image data encoded in the lower layer.

Means for Solving the Problem

An encoding apparatus of a first embodiment of the present invention isan encoding apparatus for encoding progressive image data and interlaceimage data corresponding to the progressive image data, having a firstencoding unit configured to encode picture data composing the interlaceimage data to generate first encoded data and decode and recompose thefirst encoded data to generate recomposed image data, an up sampleprocessing unit configured to up sample the recomposed image datagenerated by the first encoding unit to generate image data having thesame resolution as that of the progressive image data, and a secondencoding unit configured to encode the picture data composing theprogressive image data using the image data generated by the up sampleprocessing unit as predictive image data to generate second encodeddata.

Further, an encoding apparatus of the present invention is an encodingapparatus for encoding progressive image data and interlace image datacorresponding to the progressive image data, having a first encodingmeans for encoding picture data composing the interlace image data togenerate first encoded data and decoding and recomposing the firstencoded data to generate recomposed image data, an up sample processingmeans for up sampling the recomposed image data generated by the firstencoding means to generate image data having the same resolution as thatof the progressive image data, and a second encoding means for encodingpicture data composing the progressive image data using the image datagenerated by the up sample processing means as predictive image data togenerate second encoded data.

An encoding method of a second embodiment of the present invention is anencoding method for encoding progressive image data and interlace imagedata corresponding to the progressive image data, having a first step ofencoding picture data composing the interlace image data to generatefirst encoded data and decoding and recomposing the first encoded datato generate recomposed image data, a second step of up sampling therecomposed image data generated at the first step to generate image datahaving the same resolution as that of the progressive image data, and athird step of encoding the picture data composing the progressive imagedata using the image data generated at the second step as predictiveimage data to generate second encoded data.

A program of a third embodiment of the present invention is a programexecuted by a computer for encoding progressive image data and interlaceimage data corresponding to the progressive image data, which makes thecomputer execute a first routine of encoding the picture data composingthe interlace image data to generate first encoded data and decoding andrecomposing the first encoded data to generate recomposed image data, asecond routine of up sampling the recomposed image data generated at thefirst routine to generate image data having the same resolution as thatof the progressive image data, and a third routine of encoding thepicture data composing the progressive image data using the image datagenerated in the second routine as predictive image data to generatesecond encoded data.

A decoding apparatus of a fourth embodiment of the present invention isa decoding apparatus for decoding first encoded data obtained byencoding progressive image data and second encoded data obtained byencoding interlace image data corresponding to the progressive imagedata, having a first decoding unit configured to decode the secondencoded data, an up sample processing unit configured to interpolatefirst predictive image data generated by the decoding in the firstdecoding unit to generate second predictive image data, and a seconddecoding unit configured to decode the first encoded data based on thesecond predictive image data generated at the up sample processing unit.

Further, according to the present invention, there is provided adecoding apparatus for decoding first encoded data obtained by encodingprogressive image data and second encoded data obtained by encodinginterlace image data corresponding to the progressive image data, havinga first decoding means for decoding the second encoded data, an upsample processing means for interpolating first predictive image datagenerated by the decoding in the first decoding means to generate secondpredictive image data, and a second decoding means for decoding thefirst encoded data based on the second predictive image data generatedat the up sample processing means.

A decoding method of a fifth embodiment of the present invention is adecoding method for decoding first encoded data obtained by encodingprogressive image data and second encoded data obtained by encodinginterlace image data corresponding to the progressive image data, havinga first step of decoding the second encoded data, a second step ofinterpolating the first predictive image data generated by the decodingat the first step to generate second predictive image data, and a thirdstep of decoding the first encoded data based on the second predictiveimage data generated at the second step.

A program of a sixth embodiment of the present invention is a programexecuted by a computer for decoding first encoded data obtained byencoding progressive image data and second encoded data obtained byencoding interlace image data corresponding to the progressive imagedata, which makes the computer execute a first routine of decoding thesecond encoded data, a second routine of interpolating the firstpredictive image data generated by the decoding at the first routine togenerate second predictive image data, and a third routine of decodingthe first encoded data based on the second predictive image datagenerated at the second routine.

Effect of the Invention

According to the present invention, it is possible to provide anencoding device, an encoding method, and a program enabling encoding ofprogressive image data in the upper layer and encoding of interlaceimage data in the lower layer when performing layered encoding.

Further, according to the present invention, it is possible to provide adecoding device, a decoding method, and a program able to decode theprogressive image data encoded in the upper layer and the interlaceimage data encoded in the lower layer.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing an example of the configuration of anencoding and/or decoding system of an embodiment of the presentinvention.

FIG. 2 is a block diagram of an encoding apparatus in the encodingand/or decoding system shown in FIG. 1.

FIG. 3 is a diagram for explaining progressive image data and interlaceimage data shown in FIG. 2.

FIG. 4 is a diagram for explaining an example of the configuration of alower layer encoding circuit shown in FIG. 2.

FIG. 5 is a diagram for explaining an example of the configuration of atransform circuit shown in FIG. 2.

FIGS. 6A and 6B are diagrams for explaining an example of processing ofan intra-field predictive image generation circuit shown in FIG. 5.

FIG. 7 is a flow chart for explaining an example of the processing of anintra-frame predictive image generation circuit shown in FIG. 5.

FIGS. 8A and 8B are diagrams for explaining an example of the processingof the intra-field predictive image generation circuit shown in FIG. 5.

FIGS. 9A and 9B are diagrams for explaining an example of the processingof the intra-frame predictive image generation circuit shown in FIG. 5.

FIG. 10 is a flow chart for explaining an example of the processing ofthe intra-frame predictive image generation circuit shown in FIG. 5.

FIG. 11 is a flow chart for explaining an example of the processing ofthe intra-frame predictive image generation circuit shown in FIG. 5.

FIG. 12 is a flow chart for explaining an example of the processing ofthe intra-frame predictive image generation circuit shown in FIG. 5.

FIG. 13 is a diagram showing an example of the configuration of an upperlayer encoding circuit shown in FIG. 2.

FIG. 14 is a diagram showing an example of the configuration of a lowerlayer prediction circuit shown in FIG. 13.

FIG. 15 is a diagram showing an example of the configuration of adecoding apparatus in the encoding and/or decoding system shown in FIG.1.

FIG. 16 is a diagram showing an example of the configuration of a lowerlayer decoding circuit shown in FIG. 15.

FIG. 17 is a diagram showing an example of the configuration of atransform circuit shown in FIG. 15.

FIG. 18 is a diagram showing an example of the configuration of an upperlayer decoding circuit shown in FIG. 15.

FIG. 19 is a diagram for explaining a modification of the embodiment ofthe present invention.

DESCRIPTION of notations

1 . . . encoding and/or decoding system, 2 . . . encoding device, 3 . .. decoding device, 10 . . . layering circuit, 11 . . . delay circuit, 12. . . lower layer encoding circuit, 13 . . . transform circuit, 14 . . .upper layer encoding circuit, 15 . . . multiplex circuit, 21 . . .intra-field predictive image generation circuit, 22 . . . intra-framepredictive image generation circuit, 23, 123 . . . screen rearrangementcircuits, 31, 131 . . . processing circuits, 32, 132 . . . orthogonaltransform circuits, 33, 133 . . . quantization circuits, 34, 134 . . .rate control circuits, 35, 135 . . . reversible encoding circuits, 36,136 . . . buffer memories, 37, 137 . . . inverse quantization circuits,38, 138 . . . inverse orthogonal transform circuits, 39, 139 . . . addercircuits, 40, 140 . . . deblock filters, 41, 141 . . . frame memories,42, 142 . . . intra-prediction circuits, 43, 143 . . . motion predictionand/or compensation circuits, 51 . . . demultiplex circuit, 52 . . .delay circuit, 53 . . . lower layer decoding circuit, 54 . . . transformcircuit, 55 . . . upper layer decoding circuit, 56 . . . recomposingcircuit, 60, 160 . . . storage buffers, 61, 161 . . . reversibledecoding circuits, 62, 162 . . . inverse quantization circuits, 63, 163. . . inverse orthogonal transform circuits, 64, 164 . . . addercircuits, 65, 165 . . . deblock filters, 66, 166 . . . frame memories,67, 167 . . . screen rearrangement buffers, 69, 169 . . .intra-prediction circuits, 70, 170 . . . motion prediction and/orcompensation circuits, and 145 . . . lower layer prediction circuit.

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1 is a conceptual diagram of an encoding and/or decoding system ofthe present embodiment.

An encoding and/or decoding system 1 has an encoding apparatus 2provided on a transmission side and a decoding apparatus 3 provided on areception side.

In the encoding and/or decoding system 1, the encoding apparatus 2 onthe transmission side compresses (encodes) image data by a DiscreteCosine Transform (DCT), Karunen Loewe transform, or other orthogonaltransform and motion compensation to generate frame encoded image data(bit stream), modulates the frame encoded image data, then transmits thesame via a satellite broadcast wave, cable TV network, telephone linenetwork, mobile phone line network, or other a transmission medium 5.

On the reception side, the decoding apparatus 3 receives the encodedimage signal, demodulates it, then expands it by an inverse transform tothe orthogonal transform at the time of the above modulation and motioncompensation to generate frame image data for use.

The transmission medium 5 is not limited to the communication networksexplained above and may be an optical disc, magnetic disc, semiconductormemory, or other storage medium.

In the present embodiment, for example, as shown in FIG. 2, a layeringcircuit 10 is configured to generate progressive image data (progressiveimage signal) S10_(—)1 and interlace image data S10_(—)2 based on imagedata S9 to be encoded.

A lower layer encoding circuit 12 is configured to encode the interlaceimage data S10_(—)2 to generate lower layer encoded data S12.

A transform circuit 13 is configured to interpolate (up sample) a lowerlayer predictive image data L_PRE generated by the lower layer encodingcircuit 12 to generate lower layer predictive image data L_PREb havingthe same resolution (number of scanning lines) as that of theprogressive image data S10_(—)1 and output this to an upper layerencoding circuit 14.

The upper layer encoding circuit 14 is configured to encode theprogressive image data S10_(—)1 based on the lower layer predictiveimage data L_PREb to generate upper layer encoded data S14.

<Encoding Device>

Referring to FIG. 1, the encoding apparatus 2 will be explained.

FIG. 2 is a diagram showing an example of the overall configuration ofthe encoding apparatus 2.

The encoding apparatus 2 has for example a layering circuit 10, delaycircuit 11, lower layer encoding circuit 12, transform circuit 13, upperlayer encoding circuit 14, and multiplex circuit 15.

[Layering Circuit]

The layering circuit 10 is, for example as shown in FIG. 2, configuredto generate the progressive image data progressive image signal)S10_(—)1 and the interlace image data S10_(—)2 based on the image dataS9 to be encoded. Further, the layering circuit 10 is configured towrite the picture data FR1 to FR6 composing the above generatedprogressive image data S10_(—)1 into the delay circuit 11.

The progressive image data S10_(—)1 is for example 60 frames/sec (60p).

Further, the layering circuit 10 is configured to output the picturedata FI1 to FI6 composing the above generated index image data S10_(—)2to the lower layer encoding circuit 12.

The interlace image data S10_(—)2 is for example 60 fields/sec (60i).

[Delay Circuit]

The delay circuit 11 is configured to delay the picture data composingthe progressive image data (progressive signal) S10_(—)1 input from thelayering circuit 10 by for example exactly the processing time in thelower layer encoding circuit 12 and transform circuit 13 and output theresult to the upper layer encoding circuit 14.

[Lower Layer Encoding Circuit]

The lower layer encoding circuit 12 is configured to encode theinterlace image data S10_(—)2 input from the layering circuit 10 togenerate the lower layer encoded data S12 and output this to themultiplex circuit 15. Further, the lower layer encoding circuit 12 isconfigured to generate the lower predictive image data L_PRE in theabove encoding and output this to the transform circuit 13.

FIG. 4 is a diagram showing an example of the configuration of the lowerlayer encoding circuit 12.

The lower layer encoding circuit 12 has for example a screenrearrangement circuit 23, processing circuit 31, orthogonal transformcircuit 32, quantization circuit 33, rate control circuit 34, reversibleencoding circuit 35, buffer memory 36, inverse quantization circuit 37,inverse orthogonal transform circuit 38, adder circuit 39, deblockfilter 40, frame memory 41, intra-prediction circuit 42, and motionprediction and/or compensation circuit 43.

The screen rearrangement circuit 23 is for example configured torearrange the progressive image data S10_(—)2 input from the layeringcircuit 10 shown in FIG. 2 to a sequence of encoding in accordance witha GOP (Group Of Pictures) structure comprised by picture types I, P, andB and output the same to the processing circuit 31, intra-predictioncircuit 42, and motion prediction and/or compensation circuit 43.

The processing circuit 31 is configured to generate image dataindicating a difference between the picture data to be encoded inputfrom the screen rearrangement circuit 23 and the predictive image dataPI input from the intra-prediction circuit 42 or motion predictionand/or compensation circuit 43 and output this to the orthogonaltransform circuit 32.

The orthogonal transform circuit 32 is configured to apply a DiscreteCosine Transform (DCT), Karunen Loewe transform, or other orthogonaltransform to the image data input from the processing circuit 31 togenerate image data indicating the transform coefficient (for exampleDCT coefficient) and output this to the quantization circuit 33.

The quantization circuit 33 is configured to quantize the image data(transform coefficient before quantization) input from the orthogonaltransform circuit 32 based on a quantization scale QS input from therate control circuit 34 to generate image data indicating the transformcoefficient after the quantization and output this to the reversibleencoding circuit 35 and the inverse quantization circuit 37.

The rate control circuit 34 is for example configured to generate thequantization scale QS based on the image data read out from the buffermemory 36 and output this to the quantization circuit 33.

The reversible encoding circuit 35 is configured to store the image dataobtained by variable length encoding of the image data input from thequantization circuit 33 in the buffer memory 36. Further, the reversibleencoding circuit 35 is configured to store a motion vector MV input fromthe motion prediction and/or compensation circuit 43 or a differentialmotion vector thereof, discrimination data of reference image data, andan intra-prediction mode input from the intra-prediction circuit 42 inheader data etc.

The image data stored in the buffer memory 36 is read out as the lowerencoded data S12 to the multiplex circuit 15 shown in FIG. 2.

The inverse quantization circuit 37 is configured to apply the inversequantization processing corresponding to the quantization of thequantization circuit 33 to the image data from the quantization circuit33, generate the data obtained by that, and output this to the inverseorthogonal transform circuit 38.

The inverse orthogonal transform circuit 38 is configured to apply theinverse transform to the orthogonal transform in the orthogonaltransform circuit 32 to the data input from the inverse quantizationcircuit 37 and output the thus generated image data to the adder circuit39.

The adder circuit 39 is configured to add the image data input (decoded)from the inverse orthogonal transform circuit 38 and the predictiveimage data PI input from the intra-prediction circuit 42 or the motionprediction and/or compensation circuit 43 and output this to the deblockfilter 40.

The deblock filter 40 is configured to eliminate block distortion of thereference picture data input from the adder circuit 39 and write theresult into the frame memory 41.

The reference picture data written in the frame memory 41 is read outfrom the frame memory 41 as the lower predictive image data L_PRE andoutput to for example the transform circuit 13 shown in FIG. 2.

The intra-prediction circuit 42 is configured to determine theintra-prediction mode and block size of the prediction block so that aresidue becomes the minimum in a macro block to be intra-coded.

The intra-prediction circuit 42 uses 4×4 and 16×16 pixels as the blocksize.

The intra-prediction circuit 42 is configured to output predictive imagedata PI according to intra-prediction to the processing circuit 31 andthe adder circuit 39 when intra-prediction is selected.

The motion prediction and/or compensation circuit 43 is configured toperform the motion prediction based on the reference picture data REFwhich has already been encoded, then locally decoded and stored in theframe memory 41 and determine the motion vector and the block size ofthe motion compensation for minimizing the residue.

The motion prediction and/or compensation circuit 43 uses, as the blocksize, 16×16, 16×8, 8×16, 8×8, 8×4, 4×8, and 4×4 pixels.

The motion prediction and/or compensation circuit 43 is configured tooutput predictive image data PI according to inter-prediction to theprocessing circuit 31 and the adder circuit 39 when inter-prediction isselected.

[Transform Circuit]

Referring to FIG. 2, the transform circuit 13 will be explained.

The transform circuit 13 is configured to interpolate the lowerpredictive image data L_PRE input from the lower layer encoding circuit12, generate the lower predictive image data L_PREb having the sameresolution (number of scanning lines) as that of the progressive imagedata S10_(—)1 input from the delay circuit 11 by the upper layerencoding circuit 14, and output this to the upper layer encoding circuit14.

FIG. 5 is a diagram showing an example of the configuration of thetransform circuit 13.

The transform circuit 13 has for example an intra-field predictive imagegeneration circuit 21 and intra-frame predictive image generationcircuit 22.

Before the explanation of the processing content of the transformcircuit 13, the terms used in the present embodiment will be defined asfollows.

For example, field picture data Y of the interlace image data S10_(—)2having the same time stamp as that of a frame picture data X of theprogressive image data S10_(—)1 will be called the “base picture data ofthe picture data X”. In the example of FIG. 3, the field picture dataFI1 is the base picture data of the frame picture data FR1.

One base block MBY corresponding to the macro block MBX of the framepicture data X of the progressive image data S10_(—)1 is present in thebase picture data of the frame picture data X, that is, the fieldpicture data Y.

The base block MBY has a width of the same length and a height of halfthat of the macro block MBX.

Here, where the pixel is located at a spatial position (xP, yP) of thetop left pixel position of the base block MBX relative to the top leftpixel position of the frame picture data X, the top left pixel positionof the base block MBY is located at a spatial position (xP, yP/2).

Top field data and bottom field data continuing after that will becalled a “complementary field pair”.

In one complementary field pair, the top field data is the complementaryfield data of the bottom field data in the same pair, and the bottomfield data is the complementary field data of the top field data in thesame pair. For example, in FIG. 3, the complementary field pair iscomprised of the field data FI1 and FI2, the field data FI2 is thecomplementary field data of the field data FI1, and the field data FI1is the complementary field data of the field data FI2.

The field data A and B composing a complementary field pair will beconsidered next.

In a case where the top left pixel position of a block MBA has a spatialposition (xP_A, yP_A) relative to the top left pixel position of thefield data A, and the top left pixel position of a block MBB has aspatial position (xP_B, yP_B) relative to the top left pixel position ofthe field data B, where the blocks MBA and MBB have the same size, andthe spatial position (xP_A, yP_A) and the spatial position (xP_B, yP_B)are the same, the block MBA is the complementary block of the block MBB,and the block MBB is the complementary block of the block MBA.

When the lower predictive image data L_PRE input from the lower layerencoding circuit 12 is the top field data, the intra-field predictiveimage generation circuit 21 performs the up sample processing for thebase block data in the lower predictive image data L_PRE correspondingto the macro block MB to be encoded (current) using only the top fielddata thereof as shown in FIG. 6A to generate the lower predictive imagedata L_PREb(FI) having the same resolution as that of the progressiveimage data S10_(—)1.

When the lower predictive image data L_PRE input from the lower layerencoding circuit 12 is the bottom field data, the intra-field predictiveimage generation circuit 21 performs the up sample processing of thebase block data in the lower predictive image data L_PRE correspondingto the macro block MB to be encoded using only the bottom field datathereof as shown in FIG. 6B to generate the lower predictive image dataL_PREb(FI) having the same resolution as that of the progressive imagedata S10_(—)1.

FIG. 7 is a flow chart for explaining an example of the processing ofthe intra-field predictive image generation circuit 21 shown in FIG. 5.The intra-field predictive image generation circuit 21 is configured toperform the following processing.

In the following explanation, the number of lines of the base block datain the lower predictive image data L_PRE is defined as N. Further, thei-th line data of the base block data is defined as Base[i]. Here, i isan integer of “0” to “N−1”.

Steps ST12 to ST15 shown in FIG. 7 indicate the up sample processing ofthe top field shown in FIG. 6A and FIG. 8A, and steps ST16 to ST19 shownin FIG. 7 indicate the up sample processing of the bottom field shown inFIG. 6B and FIG. 8B.

Step ST11:

The intra-field predictive image generation circuit 21 proceeds to stepST12 when the lower predictive image data L_PRE input from the lowerlayer encoding circuit 12 is the top field data and proceeds to stepST16 when the data is not the top field data.

Step ST12:

The intra-field predictive image generation circuit 21 copies the i-thline data Base[i] of the base block data to the 2i-th line data PRED[2i] of the corresponding prediction block data in the lower predictiveimage data L_PREb.

Namely, the intra-field predictive image generation circuit 21 generatesline data PRED [0], [2], . . . , [2N−2] by the above copy.

Step ST13:

The intra-field predictive image generation circuit 21 defines the linedata lower than the lowermost line of the above base block data by oneas the line data Base[N].

Step ST14:

The intra-field predictive image generation circuit 21 stores the linedata Base[N] obtained at step ST13 in the line data PRED[2N].

Step ST15:

The intra-field predictive image generation circuit 21 adds the linedata PRED[2i] multiplied with a weighting w1 and the line dataPRED[2i+2] multiplied with a weighting w2 according to the followingEquation (1) to generate the 2i+1-th line data PRED[2i+1] of thecorresponding prediction block data in the lower predictive image dataL_PREb.

PRED[2i+1]=w1*PRED[2i]+w2*PRED[2i+2]  (1)

Step ST16:

The intra-field predictive image generation circuit 21 copies the i-thline data Base[i] of the base block data to the 2i+1-th line dataPRED[2i+1] of the corresponding prediction block data in the lowerpredictive image data L_PREb.

Namely, the intra-field predictive image generation circuit 21 generatesline data PRED[1], [3], . . . , [2N−1] by the above copy.

Step ST17:

The intra-field predictive image generation circuit 21 defines the linedata one higher than the uppermost line of the above base block data asthe line data Base[−1].

Step ST18:

The intra-field predictive image generation circuit 21 stores the linedata Base[−1] obtained at step ST17 in the line data PRED[−1].

Step ST19:

The intra-field predictive image generation circuit 21 adds the linedata PRED[2i−1] multiplied with the weighting w1 and the line dataPRED[2i+1] multiplied with the weighting w2 according to the followingEquation (2) to generate the 2i-th line data PRED[2i] of thecorresponding prediction block data in the lower predictive image dataL_PREb.

PRED[2i]=w1*PRED[2i−1]+w2*PRED[2i+1]  (2)

Step ST20:

The intra-field predictive image generation circuit 21 outputs the linedata PRED[i] generated at step ST15 and step ST19 as the lowerpredictive image data L_PREb(FI) to the upper layer encoding circuit 14.

Referring to FIG. 5, the intra-frame predictive image generation circuit22 will be explained.

When the lower predictive image data L_PRE input from the lower layerencoding circuit 12 is the top field data, as shown in FIG. 9A, theintra-frame predictive image generation circuit 22 is configured toperform the up sample processing using the base block data Base in thelower predictive image data L_PRE (top field data) corresponding to themacro block MB to be encoded (current) and complementary block data Comp(bottom field data) of the base block data to generate lower predictiveimage data L_PREb(FR) having the same resolution as that of theprogressive image data S10_(—)1.

Further, when the lower predictive image data L_PRE input from the lowerlayer encoding circuit 12 is the bottom field data, as shown in FIG. 9B,the intra-frame predictive image generation circuit 22 is configured toperform the up sample processing using the base block data Base in thelower predictive image data L_PRE (bottom field data) corresponding tothe macro block MB to be encoded (current) and complementary block dataComp (top field data) of the base block data to generate the lowerpredictive image data L_PREb(FR) having the same resolution as that ofthe progressive image data S10_(—)1.

FIG. 10 is a flow chart for explaining an example of the processing ofthe intra-frame predictive image generation circuit 22 shown in FIG. 5.

In the following explanation, the number of lines of the base block datain the lower predictive image data L_PRE is defined as N. Further, thei-th line data of the base block data is defined as Base[i]. Here, i isan integer of “0” to “N−1”.

Steps ST32 to ST36 shown in FIG. 10 indicate the up sample processing ofthe top field shown in FIG. 9A and FIG. 11, and steps ST37 to ST41 shownin FIG. 10 indicate the up sample processing of the bottom field shownin FIG. 9B and FIG. 12.

Step ST31:

The intra-frame predictive image generation circuit 22 proceeds to stepST32 when the lower predictive image data L_PRE input from the lowerlayer encoding circuit 12 is the top field data, and proceeds to stepST37 when the data is not the top field data.

Step ST32:

The intra-frame predictive image generation circuit 22 copies the i-thline data Base[i] of the base block data to the 2i-th line data PRED[2i] of the corresponding prediction block data in the lower predictiveimage data L_PREb.

Namely, the intra-frame predictive image generation circuit 22 generatesline data PRED [0], [2], . . . , [2N−2] by the above copy.

Step ST33:

The intra-frame predictive image generation circuit 22 enters the i-thline data Comp[i] of the complementary block data corresponding to thebase block data into the 2i+1-th line data PRED[2i+1].

Step ST34:

The intra-field predictive image generation circuit 21 defines line dataone lower than the lowermost line of the above base block data as theline data Base[N].

Step ST35:

The intra-field predictive image generation circuit 21 stores the linedata Base[N] obtained at step ST34 in the line data PRED[2N].

Step ST36:

The intra-field predictive image generation circuit 21 adds the linedata PRED[2i] multiplied with the weighting w1, the line data PRED[2i+2]multiplied with the weighting w2, and the line data PRED[2i+1]multiplied with the weighting w3 according to the following Equation (3)to generate the 2i+1-th line data PRED[2i+1] of the correspondingprediction block data in the lower predictive image data L_PREb.

PRED[2i+1]=w1*PRED[2i]+w2*PRED[2i+2]+w3*PRED[2i+1]  (3)

Step ST37:

The intra-field predictive image generation circuit 21 copies the i-thline data Base[i] of the base block data to the 2i+1-th line dataPRED[2i+1] of the corresponding prediction block data in the lowerpredictive image data L_PREb.

Namely, the intra-field predictive image generation circuit 21 generatesline data PRED[1], [3], . . . , [2N−1] by the above copy.

Step ST38:

The intra-frame predictive image generation circuit 22 substitute thei-th line data Comp[i] of the complementary block data corresponding tothe base block data in the 2i-th line data PRED[2i].

Step ST39:

The intra-frame predictive image generation circuit 22 defines line dataone higher than the uppermost line of the above base block data as theline data Base[−1].

Step ST40:

The intra-frame predictive image generation circuit 22 stores the linedata Base[−1] obtained at step ST39 in the line data PRED[−1].

Step ST41:

The intra-frame predictive image generation circuit 22 adds the linedata PRED[2i−1] multiplied with the weighting w1, the line dataPRED[2i+1] multiplied with the weighting w2, and the line data PRED[2i]multiplied with the weighting w3 according to the following Equation (4)to generate the 2i-th line data PRED[2i] of the corresponding predictionblock data in the lower predictive image data L_PREb.

PRED[2i]=w1*PRED[2i−1]+w2*PRED[2i+1]+w3*PRED[2i]  (4)

Step ST42:

The intra-frame predictive image generation circuit 22 outputs the linedata PRED[i] of the prediction block data generated at step ST36 andstep ST41 as the lower predictive image data L_PREb(FR) to the upperlayer encoding circuit 14.

[Upper Layer Encoding Circuit]

The upper layer encoding circuit 14 is configured to encode theprogressive image data S10_(—)1 using the lower predictive image dataL_PREb generated in the lower layer encoding circuit 12 input from thetransform circuit 13 to generate the upper encoded data S14.

FIG. 13 is a diagram showing an example of the configuration of theupper layer encoding circuit 14.

The upper layer encoding circuit 14 has for example a screenrearrangement circuit 123, processing circuit 131, orthogonal transformcircuit 132, quantization circuit 133, rate control circuit 134,reversible encoding circuit 135, buffer memory 136, inverse quantizationcircuit 137, inverse orthogonal transform circuit 138, adder circuit139, deblock filter 140, frame memory 141, intra-prediction circuit 142,and motion prediction and/or compensation circuit 143.

The screen rearrangement circuit 123 is for example configured torearrange the picture data of the progressive image data S10_(—)1 readout from the delay circuit 11 shown in FIG. 2 to the sequence ofencoding in accordance with the GOP structure comprised by picture typesI, P, and B, and output the same as the picture data ORG to be encodedto the processing circuit 131, the intra-prediction circuit 142, and themotion prediction and/or compensation circuit 143.

The processing circuit 131 is configured to generate image dataindicating the difference between the picture data to be encoded inputfrom the screen rearrangement circuit 123 and the predictive image dataPI input from the motion prediction and/or compensation circuit 143 orthe lower layer prediction circuit 145 and output this to the orthogonaltransform circuit 132.

The orthogonal transform circuit 132 is configured to apply a DiscreteCosine Transform, Karunen Loewe transform, or other orthogonal transformto the image data input from the processing circuit 131 to generateimage data indicating the transform coefficient (for example DCTcoefficient) and output this to the quantization circuit 133.

The quantization circuit 133 is configured to quantize the image data(transform coefficient before quantization) input from the orthogonaltransform circuit 132 based on the quantization scale QS input from therate control circuit 134 to generate the image data indicating thetransform coefficient after quantization, and output this to thereversible encoding circuit 135 and the inverse quantization circuit137.

The rate control circuit 134 is for example configured to generate thequantization scale QS based on the image data read out from the buffermemory 136, and output this to the quantization circuit 133.

The reversible encoding circuit 135 is configured to store the imagedata obtained by variable length encoding of the image data input fromthe quantization circuit 133 in the buffer memory 136. At this time, thereversible encoding circuit 135 stores attribute data EisTop and ETimeinput from the layering circuit 10 in the header data etc. Further, thereversible encoding circuit 135 stores the motion vector MV input fromthe motion prediction and/or compensation circuit 143 or thedifferential motion vector thereof, the discrimination data of thereference image data, and the intra-prediction mode input from theintra-prediction circuit 142 in the header data etc.

The image data stored in the buffer memory 136 is read out as the upperencoded data S14 to the multiplex circuit 15 shown in FIG. 2.

The inverse quantization circuit 137 is configured to apply the inversequantization processing corresponding to the quantization of thequantization circuit 133 to the image data from the quantization circuit133, generate the data obtained by that, and output this to the inverseorthogonal transform circuit 138.

The inverse orthogonal transform circuit 138 is configured to apply theinverse transform to the orthogonal transform in the orthogonaltransform circuit 132 to the data input from the inverse quantizationcircuit 137 and output the thus generated image data to the addercircuit 139.

The adder circuit 139 adds the image data input (decoded) from theinverse orthogonal transform circuit 138 and the predictive image dataPI input from the intra-prediction circuit 142 or the motion predictionand/or compensation circuit 143 to generate the reference (recomposed)picture data and output this to the deblock filter 40.

The deblock filter 140 is configured to eliminate the block distortionof the reference picture data input from the adder circuit 139 and writethe same into the frame memory 141.

The intra-prediction circuit 142 is configured to determine theintra-prediction mode and the block size of the prediction block so thatthe residue becomes the minimum in the macro block to be intra-coded.The intra-prediction circuit 142 uses 4×4 and 16×16 pixels as the blocksize. The intra-prediction circuit 142 is configured to output thepredictive image data PI by the intra-prediction to the processingcircuit 131 and the adder circuit 139 where the intra-prediction isselected.

The motion prediction and/or compensation circuit 143 is configured toperform the motion prediction based on the reference picture data PEFalready encoded, then locally decoded and stored in the frame memory 131and determine the motion vector and the block size of the motioncompensation for minimizing the difference of the picture data ORG to beencoded from the block data to be processed. The motion predictionand/or compensation circuit 143 uses for example any of 16×16, 16×8,8×16, 8×8, 8×4, 4×8, and 4×4 pixels as the block size.

The motion prediction and/or compensation circuit 143 is configured tooutput the predictive image data PI by inter-prediction to theprocessing circuit 131 and the adder circuit 139 when inter-predictionis selected.

The lower layer prediction circuit 145 is configured to specify thepredictive image data of the picture data ORG having a smallerdifference from the block data to be processed between the lowerpredictive image data L_PREb(FI) and L_PREb(FR) input from the transformcircuit 13 shown in FIG. 2.

FIG. 14 is a diagram showing an example of the configuration of thelower layer prediction circuit 145.

The lower layer prediction circuit 145 has a subtractor circuit 81,subtractor circuit 82, and judgment circuit 83.

The subtractor circuit 81 is configured to generate difference dataindicating the difference between the corresponding pixel data, i.e.,between the lower predictive image data L_PREb(FI) input from theintra-field predictive image generation circuit 21 shown in FIG. 5 andthe block data to be processed in the picture data ORG to be encodedshown in FIG. 13, and output this to the judgment circuit 83.

The subtractor circuit 82 is configured to generate difference dataindicating the difference between the corresponding pixel data, i.e.,between the lower predictive image data L_PREb(FR) input from theintra-frame predictive image generation circuit 22 shown in FIG. 5 andthe block data to be processed in the picture data ORG to be encodedshown in FIG. 13, and output this to the judgment circuit 83.

The judgment circuit 83 is configured to store the difference data inputfrom the subtractor circuit 81 in units of block data to generate indexdata SAD(FI). Further, the judgment circuit 83 is configured to storethe difference data input from the subtractor circuit 82 in units ofblock data to generate index data SAD(FR). Then, the judgment circuit 83is configured to specify the smaller index data between the index dataSAD(FI) and the index data SAD(FR). The judgment circuit 83 isconfigured to output the lower predictive image data L_PREb(FI) orL_PREb(FR) corresponding to the above specified smaller index dataSAD(FI) or SAD(FR) to the processing circuit 131 when the lower layerprediction circuit 145 is selected.

Predictive image data PI where the difference from the picture data ORGto be encoded becomes the smallest is selected from among the predictiveimage data PI generated by the intra-prediction circuit 142, the motionprediction and/or compensation circuit 143, and the lower layerprediction circuit 145 and output to the processing circuit 131.

The upper layer encoding circuit 14 is configured to store theprediction mode data PM indicating the finally selected prediction modein the header data and encode this in the reversible encoding circuit135. The upper layer encoding circuit 14 is configured to generateprediction mode data indicating the inter-layer/intra-field predictionwhen the lower predictive image data L_PREb(FI) is selected. The upperlayer encoding circuit 14 is configured to generate prediction mode dataindicating the inter-layer/intra-frame prediction when the lowerpredictive image data L_PREb(FR) is selected.

In the example explained above, the case where the recomposed image datagenerated in the lower layer encoding circuit 12 was used as thepredictive image data of the intra-predictive encoding in the upperlayer encoding circuit 14 was exemplified, but the recomposed image dataand motion vector generated in the lower layer encoding circuit 12 maybe utilized as the predictive image data of the inter-predictionencoding and the motion vector in the upper layer encoding circuit 14.This mode may also be used as a selection candidate.

[Multiplex Circuit]

The multiplex circuit 15 is configured to multiplex the lower encodeddata S12 input from the lower layer encoding circuit 12 and the upperencoded data S14 input from the upper layer encoding circuit 14 togenerate the encoded data S2.

[Example of Operation of Encoding Device]

An example of the operation of the encoding apparatus 2 shown in FIG. 2will be explained next.

The layering circuit 10, as shown in FIG. 2, generates the progressiveimage data (progressive signal) S10_(—)1 and the interlace image dataS10_(—)2 based on the image data S9 to be encoded. The layering circuit10 outputs the picture data FR1 to FR6 composing the above generatedprogressive image data S10_(—)1 to the delay circuit 11. Further, thelayering circuit 10 outputs the above generated interlace image dataS10_(—)2 to the lower layer encoding circuit 12.

The lower layer encoding circuit 12 encodes the interlace image dataS10_(—)2 input from the layering circuit 10 to generate the lowerencoded data S12 and outputs this to the multiplex circuit 15. Further,the lower layer encoding circuit 12 generates the lower predictive imagedata L_PRE in the above encoding and outputs this to the transformcircuit 13.

The transform circuit 13 interpolates the lower predictive image dataL_PRE input from the lower layer encoding circuit 12, generates thelower predictive image data L_PREb having the same resolution (number ofscanning lines) as that of the progressive image data S10_(—)1 inputfrom the delay circuit 12 by the upper layer encoding circuit 14, andoutputs this to the upper layer encoding circuit 14.

The delay circuit 11 delays the picture data composing the progressiveimage data (progressive signal) S10_(—)1 input from the layering circuit10 by for example exactly the processing time in the lower layerencoding circuit 12 and the transform circuit 13 and outputs the resultto the upper layer encoding circuit 14.

The upper layer encoding circuit 14 encodes the progressive image dataS10_(—)1 using the lower predictive image data L_PREb generated in thelower layer encoding circuit 12 input from the transform circuit 13 togenerate the upper encoded data S14.

The multiplex circuit 15 multiplexes the lower encoded data S12 inputfrom the lower layer encoding circuit 12 and the upper encoded data S14input from the upper layer encoding circuit 14 to generate the encodeddata S2.

<Decoding Device>

FIG. 15 is a diagram showing an example of the configuration of thedecoding apparatus 3 shown in FIG. 1.

The decoding apparatus 3 has for example a demultiplex circuit 51, delaycircuit 52, lower layer decoding circuit 53, transform circuit 54, andupper layer decoding circuit 55.

[Demultiplex Circuit]

The demultiplex circuit 51 is configured to receive as input the encodeddata S2 explained above generated by the encoding apparatus 2,demultiplex this to the lower encoded data S12 and the upper encodeddata S14, output the lower encoded data S12 to the lower layer decodingcircuit 53, and write the upper encoded data S14 into the delay circuit52.

[Delay Circuit]

The delay circuit 52 is configured to delay the upper encoded data S14input from the demultiplex circuit 51 by exactly the processing time inthe lower layer decoding circuit 53 and the transform circuit 54 andoutput the result to the upper layer decoding circuit 55.

[Lower Layer Decoding Circuit]

FIG. 16 is a diagram showing an example of the configuration of thelower layer decoding circuit 53.

The lower layer decoding circuit 53 has for example a storage buffer 60,reversible decoding circuit 61, inverse quantization circuit 62, inverseorthogonal transform circuit 63, adder circuit 64, deblock filter 65,frame memory 66, screen rearrangement buffer 67, intra-predictioncircuit 69, and motion prediction and/or compensation circuit 70.

The lower encoded data S12 input from the demultiplex circuit 51 iswritten into the storage buffer 60.

The reversible decoding circuit 61 is, when judging that the macro blockMB to be processed in the lower encoded data S12 is inter-coded,configured to decode the motion vector written in the header portionthereof and output the same to the motion prediction and/or compensationcircuit 70. The reversible decoding circuit 61 is, when judging that themacro block MB to be processed in the lower encoded data S12 isintra-coded, configured to decode the intra-prediction mode informationwritten in the header portion thereof and output the same to theintra-prediction circuit 69. The reversible decoding circuit 61 isconfigured to decode the lower encoded data S12 and output the result tothe inverse quantization circuit 62. The reversible decoding circuit 61is configured to decode the prediction mode data PM included in theheader portion and output the decoded result to for example thetransform circuit 54 shown in FIG. 15.

The inverse quantization circuit 62 is configured to inversely quantizethe image data (orthogonal transform coefficient) decoded in thereversible decoding circuit 61 based on the quantization parameter inputfrom the reversible decoding circuit 61 and output the result to theinverse orthogonal transform circuit 63.

The inverse orthogonal transform circuit 63 is configured to apply theinverse orthogonal transform processing of 4×4 to the image dataorthogonal transform coefficient) input from the inverse quantizationcircuit 62 to generate the differential image data and output the resultto the adder circuit 64.

The adder circuit 64 is configured to add the predictive image data PIfrom the motion prediction and/or compensation circuit 70 or theintra-prediction circuit 69 and the differential image data from theinverse orthogonal transform circuit 63 to generate the image data andoutput this to the deblock filter 65.

The deblock filter 65 is configured to apply deblock filtering to theimage data input from the adder circuit 64 and write the decoded imagedata after processing into the frame memory 66 and the screenrearrangement buffer 67.

The decoded image data stored in the frame memory 66 is read out as thelower predictive image data L_PRE1 to the transform circuit 54 shown inFIG. 15.

The intra-prediction circuit 69 is configured to generate the predictiveimage data PI based on the intra-prediction mode input from thereversible decoding circuit 61 and the decoded image data read out fromthe frame memory 66 and output this to the adder circuit 64.

The motion prediction and/or compensation circuit 70 is configured togenerate the predictive image data PI based on the decoded image dataread out from the frame memory 66 and the motion vector input from thereversible decoding circuit 61 and output this to the adder circuit 64.

The screen rearrangement buffer 67 is configured to store the decodedimage data written from the deblock filter 65. The decoded image datastored in the screen rearrangement buffer 67 is output as the lowerdecoded image data S53 in the sequence of display.

[Transform Circuit]

Referring to FIG. 15, the transform circuit 54 will be explained.

The transform circuit 54 is configured to interpolate lower predictiveimage data L_PRE1 input from the lower layer decoding circuit 53 basedon the prediction mode data PM from the lower layer decoding circuit 53to generate lower predictive image data L_PRE1b having the sameresolution (number of scanning lines) as that of the upper encoded dataS14 read out from the delay circuit 52 by the upper layer decodingcircuit 55 and output this to the upper layer decoding circuit 55.

FIG. 17 is a diagram showing an example of the configuration of thetransform circuit 54.

The transform circuit 54 has for example an intra-field predictive imagegeneration circuit 221 and intra-frame predictive image generationcircuit 222.

The intra-field predictive image generation circuit 221 is configured toapply the interpolation explained with reference to FIG. 6 to FIG. 8with respect to the lower predictive image data L_PRE1 to generate asthe progressive image data the lower predictive image data L_PRE1b(FI)having the same resolution as that of the upper encoded data S14 whenthe prediction mode data PM indicates the inter-layer/intra-fieldprediction.

The intra-frame predictive image generation circuit 222 is configured toapply the interpolation explained with reference to FIG. 9 to FIG. 12with respect to the lower predictive image data L_PRE1 to generate asthe progressive image data the lower predictive image data L_PRE1b(FR)having the same resolution as that of the upper encoded data S14 whenthe prediction mode data PM indicates the inter-layer/intra-frameprediction.

The transform circuit 54 is configured to output the above generatedlower predictive image data L_PRE1b(FI) and L_PRE1b(FR) to the upperlayer decoding circuit 55.

[Upper Layer Decoding Circuit]

FIG. 18 is a diagram showing an example of the configuration of theupper layer decoding circuit 55.

The upper layer decoding circuit 55 has for example a storage buffer160, reversible decoding circuit 161, inverse quantization circuit 162,inverse orthogonal transform circuit 163, adder circuit 164, deblockfilter 165, frame memory 166, screen rearrangement buffer 167,intra-prediction circuit 169, motion prediction and/or compensationcircuit 170, and lower layer prediction circuit 171.

The storage buffer 160 has the upper encoded data S14 read out from thedelay circuit 52 written into it.

The reversible decoding circuit 161 is configured to decode theprediction mode data PM included in the header data.

The reversible decoding circuit 161 is configured to decode the motionvector written in the header portion of the macro block MB and outputthe same to the motion prediction and/or compensation circuit 170 whenthe prediction mode data PM indicates that the macro block MB to beprocessed in the upper encoded data S14 is inter-coded.

The reversible decoding circuit 161 is configured to decode theintra-prediction mode information written in the header portion of themacro block MB and output the same to the intra-prediction circuit 169when the macro block MB to be processed in the upper encoded data S14 isintra-coded.

The reversible decoding circuit 161 is configured to notify the factthat the prediction mode data PM indicates the inter-layer/intra-fieldprediction or inter-layer/intra-frame prediction fact to the lower layerprediction circuit 171 when it indicates the same.

The reversible decoding circuit 161 is configured to decode the upperencoded data S14 and output the result to the inverse quantizationcircuit 162.

The reversible decoding circuit 161 is configured to output theprediction mode data PM to the motion prediction and/or compensationcircuit 170, intra-prediction circuit 169, and lower layer predictioncircuit 171.

The inverse quantization circuit 162 is configured to inversely quantizethe image data (orthogonal transform coefficient) decoded in thereversible decoding circuit 161 based on the quantization parameterinput from the reversible decoding circuit 61 and output the result tothe inverse orthogonal transform circuit 163.

The inverse orthogonal transform circuit 163 is configured to apply theinverse orthogonal transform processing of 4×4 to the image data(orthogonal transform coefficient) input from the inverse quantizationcircuit 162 to generate the differential image data and output that tothe adder circuit 164.

The adder circuit 164 is configured to add the predictive image data PIfrom the motion prediction and/or compensation circuit 170,intra-prediction circuit 169, or lower layer prediction circuit 171 etc.and the differential image data from the inverse orthogonal transformcircuit 163 to generate the image data and output this to the deblockfilter 165.

The deblock filter 165 is configured to apply the deblock filtering tothe image data input from the adder circuit 164 and write the decodedimage data after processing into the frame memory 166 and the screenrearrangement buffer 167.

The intra-prediction circuit 169 is configured to generate thepredictive image data PI based on the intra-prediction mode indicated bythe prediction mode data PM input from the reversible decoding circuit161 and the decoded image data read out from the frame memory 166 andoutput this to the adder circuit 164 when normal intra-prediction isdesignated.

The motion prediction and/or compensation circuit 170 is configured togenerate the predictive image data PI based on the decoded image dataread out from the frame memory 166 and the motion vector input from thereversible decoding circuit 161 and output this to the adder circuit 164when the prediction mode data PM indicates the inter-prediction.

The lower layer prediction circuit 171 is configured to output the lowerpredictive image data L_PRE1b(FI) and L_PRE1b(FR) input from thetransform circuit 54 or the data obtained by applying predeterminedprocessing with respect to that as the predictive image data to theadder circuit 164 when the prediction mode data PM indicatesinter-layer/intra-field prediction or inter-layer/intra-frameprediction.

The screen rearrangement buffer 167 is configured to store the decodedimage data written from the deblock filter 165. It is configured so thatthe decoded image data stored in the screen rearrangement buffer 167 isoutput as the upper decoded image data S55 in the display sequence.

[Example of Operation of Decoding Device]

The demultiplex circuit 51 is configured to receive as input the encodeddata S2 explained above generated by the encoding apparatus 2,demultiplex this to the lower encoded data S12 and the upper encodeddata S14, output the lower encoded data S12 to the lower layer decodingcircuit 53, and write the upper encoded data S14 into the delay circuit52.

The delay circuit 52 is configured to delay the upper encoded data S14input from the demultiplex circuit 51 by exactly the processing time inthe lower layer decoding circuit 53 and the transform circuit 54 andoutput the same to the upper layer decoding circuit 55.

The lower layer decoding circuit 53 is configured to decode the lowerencoded data S12 to generate the lower decoded image data S53 and outputthis. Further, the lower layer decoding circuit 53 is configured togenerate the lower predictive image data L_PRE1 (FI) and L_PRE1(FR) andoutput these to the transform circuit 54.

In the transform circuit 54, the lower predictive image data L_PRE1 istransformed to the lower predictive image data L_PRE1b(FI) andL_PRE1b(FR) having (FI) and (FR) progressive resolutions and output tothe upper layer decoding circuit 55.

The upper layer decoding circuit 55 is configured to decode the upperencoded data S14 based on the lower predictive image data L_PRE1b(FI)and L_PRE1b(FR) to generate the upper decoded image data S55 and outputthis.

As explained above, according to the encoding and/or decoding system 1of the present embodiment, the encoding apparatus 2 can encode theprogressive image data S10_(—)1 in the upper layer and can encode theinterlace image data S10_(—)2 in the lower layer when performing layeredencoding.

Further, according to the encoding and/or decoding system 1, thedecoding apparatus 3 can decode the progressive image data S10_(—)1 andthe interlace image data S10_(—)2 which are layered encoded at theencoding apparatus 2.

The present invention is not limited to the above embodiment.

Namely, those skilled in the art may make various modifications,combinations, sub-combinations, and alterations to the elements of theabove embodiment insofar as they are within the technical scope of thepresent invention or within the scope of its equivalents.

For example, all or part of the functions of the encoding apparatus 2 ordecoding apparatus 3 explained above may be executed by a processingcircuit 353 such as a CPU (Central Processing Unit) according to thescript of a program PRG stored in a memory 352 as shown in FIG. 19.

In this case, the image data to be encoded or decoded is input, andprocessing results thereof are output via an interface 351.

An example of the codes newly defined in the macro block layer in theembodiment explained above is shown in the following Table 1 and Table2.

TABLE 1 macroblock_layer_in_scalable_extension( ){ C Descriptor if(base_id_plus1 ! =  0 && adaptive_prediction_flag){   base_mode_flag2 ae(v)   If( ! base_mode_flag && HalfSpatResBaseFlag &&     !intra_base_mb(CurrMbAddr))    base_mode_refinement_flag 2 ae(v)  }  if(! base_mode_flag && ! base_mode_refinement_flag){   mb_type 2 ae(v)  If(mb_type = = I_N×N && base_id_plus1 != 0)    intra_base_flag 2 ae(v) }  if(intra_base_flag && interlace_base_layer &&progressive_curr_layer){ lower layer_intra_prediction_mode 2 ae(v) }

TABLE 2 macroblock_layer_in_scalable_extension( ){ C Descriptor if(base_id_plus1 != 0 && adaptive_prediction_flag) {  base_mode_flag 2ae(v)  If( ! base_mode_flag && HalfSpatResBaseFlag &&     !intra_base_mb(CurrMbAddr))   base_mode_refinement_flag 2 ae(v)  }  if( !base_mode_flag && ! base_mode_refinement_flag){  mb_type 2 ae(v) If(mb_type = = I_N×N && base_id_plus1 != 0)   intra_base_flag 2 ae(v) }  if(intra_base_flag && interlace_base_layer  && progressive_curr_layer && frame_structure_base_block){  lower_layer_intra_prediction_mode 2 ae(v)  }

“lower#layer#intra#prediction#mode” shown in above Table 1 and Table 2is the flag data stored in the encoded data together with the predictionmode.

The flag data indicates “0” when the up sample technique shown in FIG. 6to FIG. 8 is used and indicates “1” when the up sample technique shownin FIG. 9 to FIG. 12 is used.

When the flag data does not exist, it is judged that “0” is indicated.

“ae(v)” indicates that the designated syntax is a context-adaptiveentropy code.

“base#id#plus1” is the parameter for specifying the base picture dataused for predicting the motion vector of the current picture, the pixeldata, and the differential data.

“adaptive#prediction#flag” indicates presence/absence of the syntaxelement in the macro block layer in the scalable extension and indicates“0” when there is nothing.

When “base#mode#flag” indicates “1”, mb#type of the current macro blockis indicated, while when the reference numeral is indicated, the motionvector is indicated in accordance with the corresponding base macroblock.

When “base#mode#flag” indicates “0”, “mb#type” is not estimated when“base#mode#refinement#flag” is not “1”.

When “base#mode”flag” does not exist, “base#mode”flag” is estimated asfollows.

When “base#id#plus1” indicates “0”, it is estimated that the value of“base#mode”flag” is “0”. Where “base#id#plus1” does not indicate “0”, itis estimated that the value of “base#mode”flag” is “1”.

When “base#id#plus1” is larger than “0”, and the base layer has a widthof ½ and a height of ½ of those of the current layer,HalfSpatResBaseFlag is set at “1”, while it is set at “0” in cases otherthan that.

“intra#base#mb(CurrMbAddr)” is the function for returning “1” when thebase macro block of CrarmbAddr is the I macro block and returning “1” incases other than that.

The case where “base#mode#refinement#flag” indicates “1” indicates thatthe mb#type of the current macro block and the reference numeral areestimated based on the corresponding base macro block. When“base#mode#refinement#flag” indicates “1”, a motion vector obtained byadding ¼ pixel resolution to the prediction value of the motion vectorobtained by using the motion vector of the base macro block isspecified.

When “base#mode#refinement#flag” indicates “0”, mb#type is notestimated.

“mb#type” indicates the macro block type. The semantic of “mb#type”depends upon the slice type.

A case where “intra#base#flag” indicates “1” indicates the macro blocktype I#BL.

When “intra#base#flag” does not exist, “intra#base#flag” is estimated asshown below.

When “base#mode#flag” is “1” and “mb#type” is I_N×N, it is estimatedthat “intra#base#flag” is “1”. It is estimated that “intra#base#flag” is“0” in cases other than that.

“I#N×N” indicates that the macro block type is any of intra 6×6, intra4×4, or I_BL.

“I#BL” indicates that the prediction value thereof is not the peripheralpixel data, but the intra-macro block obtained from the base picturedata.

“interlace#base#layer” indicates that the source of the base layer isthe interlace format.

“progressive#curr#layer” indicates that the source of the current layeris the progressive format.

“frame#structure#base#block” indicates that the base block is encoded bythe frame structure.

For reference, an embodiment of an encoding and/or decoding systemincluding the encoding apparatus and the decoding apparatus of thepresent invention will be explained.

The correspondence between the configuration of the encoding and/ordecoding system of the present embodiment and the configuration of thepresent invention will be explained.

The encoding apparatus 2 is an example of the encoding apparatus of thepresent invention. When the processing content (function) of theencoding apparatus 2 is described by the program PRG exemplified in FIG.19, this program PRG is an example of the program of the presentinvention. Such a program is usually stored in a storage medium ortraded via the communication channel and performs operations when loadedin a computer. Accordingly, the program of the present inventionincludes such a transaction format and operation format.

The lower layer encoding circuit 12 explained with reference to FIG. 2is an example of the first encoding means and first encoding unit of thepresent invention, the transform circuit 13 is an example of the upsample processing means and up sample processing unit, and the upperlayer encoding circuit 14 is an example of the second encoding means andsecond encoding unit.

The decoding apparatus 3 is an example of the decoding apparatus of thepresent invention. When the processing content (function) of thedecoding apparatus 3 is described by the program PRG exemplified in FIG.19, this program PRG is an example of the program of the presentinvention. Such a program is usually stored in a storage medium ortraded via a communication channel and performs operations when loadedin the computer. Accordingly, the program of the present inventionincludes such a transaction format and operation format.

The lower layer decoding circuit 53 explained with reference to FIG. 15is an example of the first decoding means and first decoding unit of thepresent invention, the transform circuit 54 is an example of the upsample processing means and up sample processing unit, and the upperlayer decoding circuit 55 is an example of the second decoding means andsecond encoding unit.

1.-10. (canceled)
 11. A decoding apparatus for decoding first encodeddata obtained by encoding progressive image data and second encoded dataobtained by encoding interlace image data corresponding to theprogressive image data, the decoding apparatus comprising: a firstdecoding unit configured to decode the second encoded data, an up sampleprocessing unit configured to interpolate first predictive image datagenerated by the decoding in the first decoding unit to generate secondpredictive image data, and a second decoding unit configured to decodethe first encoded data based on the second predictive image datagenerated at the up sample processing unit
 12. A decoding apparatus fordecoding first encoded data obtained by encoding progressive image dataand second encoded data obtained by encoding interlace image datacorresponding to the progressive image data, the decoding apparatuscomprising: a first decoding means for decoding the second encoded data,an up sample processing means for interpolating first predictive imagedata generated by the decoding in the first decoding means to generatesecond predictive image data, and a second decoding means for decodingthe first encoded data based on the second predictive image datagenerated at the up sample processing means
 13. A decoding method fordecoding first encoded data obtained by encoding progressive image dataand second encoded data obtained by encoding interlace image datacorresponding to the progressive image data, the decoding methodincluding: a first step of decoding the second encoded data, a secondstep of interpolating the first predictive image data generated by thedecoding at the first step to generate second predictive image data, anda third step of decoding the first encoded data based on the secondpredictive image data generated at the second step 14.-15. (canceled)